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I.  INTRODUCTION 


Three  commonly  (but  not  consistently)  used  terms  in  the  control 
literature  are  "performance  measure",  "performance  index",  and 
"performance  criterion".  The  reader  should  note  that  by  "performance 
measure"  we  mean  a  functional  that  assigns  a  nonnegative  real  number  to 
each  sample  run  of  the  control  system.  Thus,  for  stochastic  systems 
performance  measures  are  random  variables. 

We  call  statistical  quantities  associated  with  performance  measures 
for  the  sake  of  optimization  "performance  indices".  In  this  paper 
optimization  consists  of  minimizing  weighted  sums  of  performance  indices, 
and  "performance  criterion"  is  the  term  we  use  to  refer  to  the  objective 
of  achieving  such  a  minimum. 

The  research  reported  in  this  paper  is  the  culmination  of 
approximately  a  decade  of  effort  orininated  by  Sain  [6],  [7],  [8] 
continued  by  Liberty  [4]  and  Sain  and  Liberty  [10],  and  completed  by 
Liberty  and  Hartwig  [5]  and  Hartwig  [2].  In  the  sequel  we  present 
new  analytical  results  in  Linear-Quadratic-Gaussian  (LQG)  control 
theory  which  reflect  a  design  philosophy  that  significantly  departs 
from  traditional  lines  of  thought. 

The  objectives  of  the  overall  research  effort  were  two  fold:  first, 
to  solve  LQG  control  problems  with  broader  criteria  than  minimum  average 
performance;  and  second,  to  develop  broad  statistical  performance  analysi 
techniques  for  LQG  systems. 

Original  work  directed  at  the  first  objective  resulted  in  the 
solution  of  the  open-loop  minimum-variance  problem  [  7],  [  9],  while 
that  directed  at  the  second  resulted  in  the  development  of  various 
partial  and  complete  statistical  descriptions  of  performance  [  4],  [  6], 


[  8],  [10]  with  the  complete  solution  of  the  performance  analysis 
problem  appearing  in  [  5]. 

These  performance  analysis  results  are  utilized  in  section  V  to 
demonstrate  the  characteristics  of  an  LQG  control  system  designed 
according  to  criteria  developed  herein. 

Although  solution  of  the  open  loop  minimum  variance  control 
problem  provided  new  insight  into  the  LQG  theory,  it  had  little  practical 
impact  on  LQG  design.  This,  of  course,  was  simply  due  to  its  inherent 
lack  of  feedback  structure. 


With  this  history  in  mind  and  with  insight  gained  from  the  work 
reported  in  [  5]  we  now  set  out  to  develop  a  general  LQG  design  philosophy 
and  an  accompanying  design  procedure. 


The  following  objectives  form  th* framework  of  our  development: 


i.  We  require  that  the  performance  criteria  we  decide  / 
upon  yield  linear  control  laws.  This  is  desirable 
for  implementation  considerations  and  will 
guarantee  that  the  final  system  will  still  lie  in 
the  LQG  class  thus  assuring  the  applicability  of 
the  design  performance  analysis  results  of  [  5]. 


ii. 


iii . 


iv. 


We  demand  physically  realizable  controllers.  (It 
may  seem  silly  to  even  state  this  objective,  but 
actually  one  must  carefully  guard  against  non¬ 
causality  that  can  artificially  arise  in 
optimization. ) 

We  do  not  want  to  sacrifice  any  more  computational 
tractability  than  necessary  for  the  sake  of 
achieving  more  general  results. 

We  want  to  select  performance  criteria  that  will 
allow  the  designer  to  affect  the  probability 
distribution  of  his  performance  measure(s)  in 
a  desirable  way.  (For  the  purpose  of  performance 
analysis  there  may  be  more  than  one  performance 
measure  for  each  control  system. ) 


✓ 


In  section  II  we  state  several  well  known  facts  from  the  LQG 


theory.  For  the  sake  of  conciseness  we  heavily  reference  the 
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tutorial  paper  by  Tse  [11 ]  and  use  his  notation  as  much  as  possible. 
This  section  also  contains  the  definition  of  quadratic  performance 
measures  to  be  utilized  in  the  control  system  design  procedure. 

Section  III  contains  the  development  of  a  complete  statistical 
description  of  performance  in  terms  of  an  "accessible  state".  In 
addition,  design  performance  indices  are  defined  in  this  section. 

In  section  IV  we  state  the  new  LQG  performance  criteria  and 
solve  a  general  problem  within  this  class.  Section  V  contains  a 
numerical  example  of  design  according  to  the  new  criteria. 


II.  SYSTEM  MODELS  AND  PERFORMANCE  MEASURES 

Let  Rn  denote  the  n-fold  Cartesian  product  of  the  real  line,  and 
let  J  denote  the  real  line  interval  [tQ,  t^].  The  linear  system  to  be 
controlled  is  described  by 

Mil  -  A(t)x(t)  +  B(t)u(t)  +  C(t),  (1) 

and 

z(t)  =  C(t)x(t)  +  0(t),  teJ  ,  (2) 

where  the  state  x(t)  e  Rn,  the  control  action  u(t)  e  Rm,  and  the 

M 

observation  process  z(t)  e  R  .  The  initial  condition  for  (1),  x(tQ), 
is  assumed  to  be  Gaussian  with  mean 

xQ  =  E{x(tQ) }  (3) 

and  covariance 

£0  =  E{[x(t0)-x0][xT(t0)-xJ]}  (4) 

where  (')  denotes  matrix  transposition.  The  state  noise  process,  t(t) 
is  zero-mean  Gaussian-white  with  covariance  kernel 

E{c(t)5T(x)}  =  =(t)6(t-x) ,  t ,teJ  (5) 

where  =(t)  is  symmetric  and  positive  semi-definite  on  J.  The 
measurement  noise  process,  e(t),  is  zero-mean  Gaussian-white  with 
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covariance  kernel 


E{e(t)e  (t)}  =  e(t)5(t-r),  t,T£j 


(6) 


where  ©(t)  is  symmetric  and  positive  definite  on  j.  For  convenience  we 
assume  that  ^(t),  e(t),  and  x(tQ)  are  all  uncorrelated.  That  is. 


EU(t)e  (x)}  =  0,  t,teJ 


(7) 


and 


E{[x(t0)-x0]c‘(t)>  =  0,  teJ, 


£a*(to)-xol3'(t)}-0,  teJ 


(8) 

(9) 


The  control  action,  u(t),  is  assumed  to  be  a  causal  function  of  the 
observation  process.  That  is. 


u ( t )  =  <|»(t,z(t);  TE[t0,  t]). 


(10) 


where  t , * )  satisfies  the  Lipschitz  condition  in  [ll].  All  matrix 
functions  including  the  mapping  ip(t , * )  are  assumed  to  be  smooth  enough 
to  guarantee  mean  square  continuity  of  the  x-process  on  J.  Consequently 
x  is  a  finite  energy  process  on  J. 

We  define  three  measures  of  system  performance.  The  first,  hence¬ 
forth  referred  to  as  the  design  performance  measure,  is  given  by 


T  f 

J  =  x  (tf)Sx(tf)  +  /  CxT(t)Q(t)x(t)  +  uT(t)R(t)u(t)]dt, 

*o 

where  the  terminal  penalty  weighting,  S,  is  symmetric  and  positive 


(ID 
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semi-definite  as  is  the  weighting  Q(t)  on  J.  The  weighting  R(t)  is 
symmetric  and  positive  definite  on  J.  We  also  assume  that  Q(t)  and 
R(t)  are  continuous  on  J.  The  performance  measure,  J  ,  describes  a-priori 
design  specifications  involving  the  relative  importance  of  state 
regulation  and  control  effort. 

The  remaining  two  measures  of  system  performance  will  be  referred 
to  as  design-analysis  performance  measures  and  are  defined  by 


xT(t)X(t)x(t)dt, 


(12) 


(13) 


The  weighting  matrices  X(t)  and  U(t)  are  assumed  to  be  symmetric  and 
positive  semi-definite  on  J.  Conceptually. Jx  measures  the  state 
regulating  quality  of  a  given  design. and  0^  measures  the  corresponding 
control  effort.  The  weightings  X(t)  and  U(t)  are  chosen  by  the 
designer  to  select  and/or  weight  particular  components  of  x  or  u  for 
analysis.  Performance  analysis  as  demonstrated  in  section  V  consists 
of  obtaining  probabilistic  or  statistical  descriptions  of  Jx  and  Ju 
after  a  design  is  achieved. 

Quantities  of  special  interest  in  our  development  are  the 
conditional  mean  and  conditional  covariance  of  x(t)  given  by. 


x(t)  =  E{x(t)|Ft),  teJ,  (14) 

and 

K*(t,x)  =E{[x(t)-x(t)3[xT(x)-xT(x)]|F0}, 

o  *  tVx,  t,  TeJ 


(15) 


where  is  the  sigma  algebra  induced  by  the  measurements  (x(t), 
t  e  [tQ,  t]  }.  When  t  =  tf  we  will  write  F  without  a  subscript.  Define 
the  single  argument  covariance  i(t)  as 

i(t)A  K~(t,  t),  teJ  •  (16) 

It  is  well  known  that  the  conditional  covariance  kernel  is  nonrandom 
[11],  and  that  x  evolves  according  to 

^  x(t)  =  A(t)x(t)  +  B(t)u(t)  +  W(t)v(t) ,  teJ  (17) 

where 

x(tQ)  =  x0,  (18) 

W(t)  =  t(t)CT(t)Q_1(t),  teJ,  (19) 

and  E(t)  satisfies 


^  l(t)  =  A(t)z(t)  +  z(t)AT(t)  +  =(t) 

-  r(t)CT(t)0-1(t)C(t)z(t),  teJ,  (20) 

with 

£(t0)  =  20-  (21) 

The  control  action,  u(t),  is  as  in  (10)  and  the  "innovations  process", 
v(t),  defined  by 

v(t)  A  z(t)  -  C(t)x(t) ,  teJ,  (22) 


is  zero-mean  Gaussian-white  with  covariance  kernel 
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E { v ( t ) vT ( t ) }  =  0(t)s(t-r),  t,xeJ  (23) 

The  process,  x,  will  be  an  "accessible  state"  in  our  formulation  and 
the  x-process  model  in  (17)  alone  plays  the  key  dynamical  role  in  the 
determination  of  u(t).  However,  both  (17)  and  (1)  are  utilized  in 
post-design  performance  analysis. 


III.  PERFORMANCE  STATISTICS  AND  PERFORMANCE  INDICES 


The  process  generated  by  (1)  is  a  nonzero-mean  Gauss-Markov 
process  which  we  expand  in  an  orthonormal  series 

oo 

x(t)  -  J  xi4>i  (t) ,  teJ  (24) 

where  the  x^  are  conditionally  Gaussian  scalar  random  variables 
given  by 

xi  =  xT(tf)S<{.i(tf)  +  /  xT(t)Q( t)<|> - (t)dt  (25) 

to 

with  conditional  means 


=  E{xT(tf)|F}S*.(tf) 
tf  T 

+  /  E{x  (t)  |F.  }Q(t)<j>i  (t)dt, 

t 

co 

In  addition,  the  x^  are  assumed  to  be  conditionally  uncorrelated, 
that  is. 


(26) 


E([xi  -  m^tXj  -  mj]|F}  =  X-6^  Vi,j. 


(27) 


The  nonrandom  vector  valued  functions,  $.j(t),  are  chosen  to  satisfy 
the  orthonormality  condition, 


Under  the  «"mptir  we  have  made  on  the  x-process,  J  is  finite 
with  probability  one;  see  Doob  [1].  It  follows  from  Parseval's  Theorem 
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that 

tf 

J  =  f  xf  +  /  uT(t)R(t)u(t)dt,  (29) 

1=1  *0 

where  convergence  is  with  probability  one;  see  Kolmogorov  and  Fomin  [3]. 

Since  the  are  conditionally  Gaussian  and  conditionally  un¬ 
correlated,  they  are  conditionally  independent  as  are  their  squares. 

2 

The  conditional  characteristic  function  of  each  term  in  (29)  is  of 
the  noncentral  chi-square  type  given  by. 


-1/2  2  -1 

C  2  (jui)  =  (1  -  2ju)Xi)  exp[jtom.(l  -  2jux.)  ].  (30) 

x  ]F 
i 

The  conditional  characteristic  function  of  J  follows  as, 

-1/2  T 

C1iv(jo))  =  [  n  (1  -  2ja)X ■ )  ]  •  exp  Ljw  J  u1  (t)R(t)u(t)dt 

J|F  i=1  1  r 


+  -  2jo,x.)-1].  (31) 

The  second  conditional  characteristic  function,  Tjjp(joj),  is  defined 
as  the  natural  logarithm  of  Cjip(ju»),  that  is. 


Tj|p(jw)  =  1  n[Cj  |  p(  jii))]. 


(32) 


The  MacLaurin  series  representation  of  Tj|p(jw)  is  given  by 


Tj|p(ju>)  =  Ki  | p 


where  the  coefficients  K^pare  called  conditional  cumulants.  Utilizing 
(30),  it  can  be  easily  shown  that  the  first  conditional  cumulant  of 
J  is  given  by 
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<1!e=  I  X<  *  I  +  /  uT(t)R(t)u(t)dt, 

'lf  •  i=l  t0 

while  the  remaining  conditional  cumulants  are  of  the  form 


k-l»  0  k-1 
r 


Kk|F  =  ^  .1  +  ^‘2  . mi xi  »  ^  >  1  • 


It  is  easily  shown  that 


k1|F=E{J|F},  (36) 

<2|F  =  E{( J  -  E{J|F})2|F},  (37) 

and  that  in  general  the  conditional  noncentral  moments,  u|<|F>  are  related 
to  the  conditional  cumulants  by 


yk+l|F  =  (j)uk-j|^j+l|F’  (38) 

The  noncentral  moments,  y^,  are  related  to  the  conditional  noncentral 
moments  by 


\  =  E{yk j  *  ' .  t39' 

The  relationship  between  the  cumulants,  and  the  conditional 
cumulants  can  be  determined  by  first  noting  that  the  noncentral 
moments  and  the  cumulants  are  related  in  a  manner  identical  to  that  for 
the  conditional  statistics,  that  is, 


=  j=o(j)yk-jVi 


(40) 
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“91/1  »T  (  k-1  )  * 

=  X  (Vs  KX  (tf»  tf)Sx(tf) 

~T  (k-1) 

+  x  (tf )S  /  (tf.  t)Q(t)x(t)dt 
to 

(k-1) 

+  /  x  (t)Q(t)K~  (t>  tf)dtSx(tf) 

to 

Hr  (k-1) 

+  /  J  x  (t)Q(t)K“  (t,T)Q(T)x(t)dxdt,  k  >  1,  (48) 

t  t 
zo  ^0 

For  the  case,  k  =  1,  it  is  obvious  that 

oo  t^ 

l  =  xT(tf)Sx(tf)  +  /  xT(t)Q(t)x(t)dt.  (49) 

1  =  1  t 

o 

Recalling  the  objectives  listed  in  section  I  we  now  examine  the 

statistical  expressions  in  (43),  (47),  (48)  and  (49). 

i.  Note  that  (48)  and  (49)  are  quadratic  in  x. 

ii.  Note  that  (47)  does  not  depend  on  u. 

iii.  Note  from  (43)  that  the  value  of  every  statistic 

of  J  is  affected  by  the  expected  value  of  the 

corresponding  conditional  cumulant. 

Thus  we  choose  the  expected  value  of  the  conditional  cumulants  of  J  as 
performance  indices.  Optimization  performed  over  these  objects  will 
yield  linear  control  laws  and  will  provide  the  design  engineer  with  a 
mechanism  for  affecting  the  probability  distribution  of  J  in  a  broad 
statistical  sense. 


IV.  NEW  PERFORMANCE  CRITERIA  FOR  LQG  CONTROL 

We  now  select  a  class  of  performance  criteria.  In  particular,  we 

choose  to  minimize  the  expected  value  of  a  weighted  sum  of  conditional 

cumulants  of  J.  This  minimization  is  to  be  done  over  the  class  of 

admissible  control  laws  described  in  [11].  Thus, our  criterion  is 
N 

min  E{  J  u  admissible,  N  finite, 

where  the  a.  are  chosen  by  the  designer  to  reflect  the  relative  importance 
of  affecting  a  particular  statistic  of  0. 

This  criterion  will  preserve  linearity  while  adding  statistical 
breadth  to  the  LQG  design  procedure.  It  should  be  noted  that  the  classical 
minimum-mean  LQG  control  problem  is  a  special  case  of  the  new  class 
of  problems. 

The  objectives  that  we  must  yet  meet  are  those  of  physical  realizability 
and  computational  tractability.  To  achieve  these  we  define  a  class  of 
dynamical  variables  all  driven  by  x.  The  resulting  dynamical  systems 
become  part  of  the  feedback  control  structure.  These  variables  are  given  by 

t  (k) 

nk(t)  4  /  K~  (t>*)Q(x)x(x)dx,  t  e  J.  (50) 

to 

This  variable  arises  naturally  in  each  conditional  cumulant  expression 
by  making  the  observation  that  the  last  term  in  (48)  contains  a  symmetric 
(in  argument)  integrand.  Thus 

'J*  V (k-1)  .  tf  t  .T  fk-1 ) 

1  I  xT(t)Q(t)K;  (t,x)Q(x)x(x)dxdt=  2/  j  x‘(t)Q(t)K;  (t»t)Q(x)x(x)dxdt 

i.  *0  *o  *0 

tf'T 

=  2  J  xT(t)Q(t)nM(t)dt.  (51) 
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It  follows  immediately  that 

j  uju}'1,  xT(tf)Sk^  (tf,tf)Sx(tf)  +  2  xT(tf)Snk_1(tf) 

+  2  /  x'(t)Q(t)nk.1(t)dt.  (52) 

to 

At  this  point  we  can  best  illustrate  the  results  by  considering 
a  single  problem  within  the  new  class.  Consider  the  problem  of 
minimizing  £{K-[jp  +  oK2jF*  *  Here  We  wish  to  decrease  the  variance 
J  at  the  expense  of  an  increase  in  the  mean  of  J.  Disregarding  the 
uncontrollable  terms,  this  criterion  may  be  explicitly  written  as 

min  E{xT(tp)Sx(t.r )  +  J[x T(t)Q( t)x(t)  +  uT(t)R(t)u(t)]dt 
u  *o 

+  4coJ(tf)Sz(t.f )Sx(t^)  +  8ax^(t^)Sn-j  (t^) 

tf 

+  8a  /  xT(t)Q(t)n-|(t)dt}  , 


where  n-j (t)  evolves  from 


n-j  (t)  =  A(t)rvl(t)  +  E(t)Q(t)x(t) ,  t  e  J 


W  =  °* 


By  defining  an  augmented  state  vector  this  criterion  may  be  rewritten  as 
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min  E{ 
u 

x(tf) 

^1  ( t^r ) 

T 

S  +  4ctSz(tf)S  4aS 

4aS  0 

x(tf) 
n-j  (t^) 

■»  " 

*  — 

x(t) 

1 

~Q(t) 

4aQ(t) 

x(t) 

.nl(t). 

4aQ(t) 

0 

n-j  (t) 

+  uT(t)R(t)u(t)dt} 


where  the  augmented  state  vector  evolves  from 


d_ 

dt 


x(t) 

j. 

A(t) 

0 

x(t) 

+ 

B(t) 

n-,  ( t) 

s(t)Q(t) 

A(t; 

n]  (t) 

0 

with 


x(t0) 


w(t) 

0 


L 


'0 

0 


v(t),  teJ 


nl(V 

However, the  solution  of  an  accessible  state  problem  with  this 
is  well  known  [11]  and  is  given  by 


u(t)  =  -R{hsJ(t)  0]M(t) 


x(t) 
n-|  ( t) 


,  teJ 


u(t) 


(55) 


(56) 

structure 


(57) 


where  M(t)  is  a  solution  of 
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jjr  M(t)  =  -M(t) 


A(t) 

E(t)Q(t) 


0 

A(t) 


AT(t) 

0 


Q(t)z(t) 

AT(t) 


M(t) 


Q(t)  4a'5(t) 

4aQ( t)  0 


+  M(t) 


B(t)R_1 

0 


(t)BT(t)  0 

0 


M(t)  ,  teJ  (58) 


with  M(tf)  = 


S  +  4aSt(tf)S 
4aS 


4aS 

0 


(59) 


Notice  that  the  resulting  feedback  structure  is  linear,  dynamical, 
and  physically  realizable.  A  block  diagram  of  the  overall  system  is 
shown  in  Figure  1,  where  M-j -j  ( t )  and  M12  (t)  are  the  obvious  n  x  n 
submatrices  of  mU).  The  next  section  contains  a  numerical  example 
with  accompanying  performance  analysis  to  illustrate  the  procedure 
and  properties  of  the  solution. 


V.  PERFORMANCE  CRITERIA  -  A  NUMERICAL  EXAMPLE 


Consider  a  scalar  system  described  by  (1)  -  (9)  with 

J  =  [0>  1],  (60) 

A(t)  -  B(t)  =  C(t)  =  xQ  -1,  te[0,  1],  (61) 

*o  =  0  (62) 

S(t)  =  0.25,  te[0,  1],  (63) 

and 

0 (t)  =  0.35,  te[0,  1].  (64) 

We  select  (11)  as  the  measure  of  system  performance  with 

S  =  0,  (65) 

and 

Q(t)  =  R(t)  =  1 ,  te[0,  1].  (66) 

We  will  use  as  our  performance  criteria, 
min  E{ k-j  i  p  +  a<2 1  p} 


The  resulting  feedback  structure  for  the  above  system  matrices  is 
given  by  (57)  -  (59). 

A  plot  of  mean  and  variance  of  the  performance  measure  is  shown  in 
Fig.  2.  As  expected,  for  an  increase  in  a  one  observes  both  an 
increase  in  the  mean  and  a  decrease  in  the  variance.  However,  there  is 
yet  an  additional  characteristic  of  these  plots  which  is  just  as 
significant.  In  the  region  of  small  a,  a  given  change  in  o  results 
in  a  relative  change  in  the  variance  which  is  greater  than  the  corresponding 
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relative  change  in  the  mean.  Hence,  for  small  a  one  obtains  a  mean  only 
slightly  larger  than  the  minimum  mean,  while  the  resulting  variance  is 
considerably  less  than  the  maximum  variance  occurring  at  a  =  0.  Of 
course,  general  conclusions  cannot  be  drawn  here,  but  our  philosophy 
is  supported. 

In  order  to  provide  a  more  complete  statistical  description  of 
system  performance,  the  probability  density  functions  of  J  for  a  =  0, 
a  =  0.5,  and  a  =  2.0  are  shown  in  Fig.  3. 

For  purposes  of  evaluating  state  regulation  and  control  effort, 
we  will  now  examine  the  resulting  behavior  of  Jx  and.Ju  given  in  (12) 
and  (13)  where  we  assume  that 

X ( t )  =  U(t)  =  1,  te[0,  1],  (67) 

Plots  of  mean  and  variance  of  both  J  and  J  are  shown,  respectively, in 
Figs.  4  and  5.  An  interesting  characteristic  of  these  curves  is  that 
for  increasing  a,  both  the  mean  and  variance  of  the  state  regulation 
decrease.  One  obviously  has  to  pay  a  price  for  this  good  performance, 
and  this  is  seen  in  the  control  effort  where  both  the  mean  and  variance 
increase  with  increasing  a.  Figs.  6  and  7  show  the  probability  density 
functions  of  J*.  and  ^  ,  respectively,  for  the  values  a  =  0,  a=  0.5, 
and  <*=  2.0.  The  real  value  of  these  density  curves  is  that  for  each 
value  of  « ,  one  can  see  probabilistically  the  resulting  trade-off 
between  state  regulation  and  control  effort. 

In  an  actual  design  the  design  engineer  would  iterate  through  the 
procedure,  varying  a,  S,  Q(t),  and  R(t)  at  each  iteration  as  needed, 
until  desirable  performances,  Jx  and  are  obtained. 


STATE  ESTIMATOR 


ie  overall  system  resulting  from  the  criteria  min  E{k,|F 
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VI.  CONCLUSION 


The  new  class  of  LQ6  performance  criteria  developed  should  enhance 
the  applicability  of  the  LQG  theory.  Experimentation  with  the  proposed 
design  procedure  should  be  carried  out.  There  are  details  on  controllability 
and  properties  of  equation  solution,  etc.  that  we  have  chosen  not  to 
address  here. 

The  line  of  thinking  that  has  led  to  these  results  may  also  prove 
to  be  useful  in  the  estimation  area, particularly  with  regard  to  the 
development  of  "robust"  linear  estimators. 
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